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Abstract — We consider communication over the binary erasure 
channel (BEC) using low-density parity-check (LDPC) codes 
and belief propagation (BP) decoding. For fixed numbers of 
BP iterations, the bit error probability approaches a limit as 
blocklength tends to infinity, and the limit is obtained via density 
evolution. On the other hand, the difference between the bit 
error probability of codes with blocklength n and that in the 
large blocklength limit is asymptotically a(e,t)/n + 9(n~ 2 ) 
where a(e,t) denotes a specific constant determined by the 
code ensemble considered, the number t of iterations, and the 
erasure probability e of the BEC. In this paper, we derive a 
set of recursive formulas which allows evaluation of the constant 
a(e, t) for standard irregular ensembles. The dominant difference 
a(e,t)/n can be considered as effects of cycle-free and single- 
cycle structures of local graphs. Furthermore, it is confirmed via 
numerical simulations that estimation of the bit error probability 
using a(e, t) is accurate even for small blocklengths. 

Index Terms — low-density parity-check codes, belief propa- 
gation, binary erasure channel, density evolution, finite-length 
analysis 



I. Introduction 

IT is well known that low-density parity-check (LDPC) 
codes for transmission over binary memoryless symmet- 
ric channels approach channel capacity with low-complexity 
iterative decoder called belief propagation (BP) decoder. Es- 
pecially for the binary erasure channels (BEC), LDPC codes 
with BP decoder provably achieve channel capacity (TJ. Large- 
blocklength limit of the bit error probability of BP decoder 
with a fixed number of iterations can be calculated by the 
method called density evolution 0. In this paper, we consider 
how fast the bit error probability approaches the limit as 
blocklength tends to infinity. Although performance analysis of 
LDPC codes is often developed for general binary-input mem- 
oryless symmetric channels @, Q, j4], 0, GD, we restrict 
our attention in this paper to the case where the channel is 
the BEC, since performance analysis on the BEC HI, Q, (8), 
|9l . iflOll is generally simpler than that for general channels. 
In density evolution, the bit error probability is calculated 
recursively by considering tree neighborhoods whose depth 
is equal to the number of iterations. In the analysis of this 
paper, we consider not only tree neighborhood graphs but also 
single-cycle neighborhood graphs in order to derive the most 
dominant term in the bit error probability which vanishes in 
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the large-blocklength limit. We would like to mention that it 
might be possible to generalize our analysis to other channels 
and iterative decoders since the approach taken in our analysis 
is based on density evolution which is applicable to any 
combination of a channel and an iterative decoder. 

Let Pb(n, e, t) denote the bit error probability of an en- 
semble of codes of blocklength n over the BEC(e) after t BP 
iterations. The large-blocklength limit of the bit error proba- 
bility after t iterations is denoted by Pb(oo, e, t). Evaluation 
of Pb(oo, e, t) using density evolution has revealed that there 
exists a threshold erasure probability 6bp such that the bit 
error probability Pb(oo,e, t) after a sufficient number of BP 
iterations tends to if e < £bp and to a strictly positive value 
if e > e B p- 

From a practical point of view, it is desirable to evaluate 
Pb{n, e,t) for a finite n, which, however, is much more 
complicated than the evaluation of Pb(oo,e,t). The bit and 
block error probabilities for finite blocklength and for infinite 
number of iterations are calculated exactly via stopping-set 
analysis for regular ensembles [7| and also for irregular 
ensembles ifTTl . Furthermore, the bit and block error prob- 
abilities of expurgated ensembles for finite blocklength and 
for finite numbers of iterations are also calculated exactly in 
a combinatorial way lfl2ll . However, these analyses require 
high computational costs which grow like a power of the 
blocklength and like an exponential of the number of degrees. 
This fact severely restricts usefulness of these analyses. 

An approach to a finite-length analysis for irregular ensem- 
bles with low computational complexity would be to consider 
large-rt asymptotics. There are two efficient methods to derive 
large-n asymptotics for the bit error probability for blocklength 
n and for infinite number of iterations, which is denoted by 
Pb{n, e, oo). The method proposed by Di, Richardson, and 
Urbanke JH has shown that the bit error probability below the 
threshold after infinite number of iterations is expressed as 
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One may thus obtain an approximation formula for 
Pb(n., e, oo) by ignoring the term o(n _1 ) in ([]]). However, 
the approximation is not accurate near the threshold for any 
irregular ensembles due to the following reasons. If the limit 
lim^^oo Pb(n, e, oo) is discontinuous at ebp as a function 
of e (i.e., limn^oo Pb(n, £bp, oo) > 0), convergence to the 
limit is not uniform since Pb{n, e, oo) for any finite n is 
continuous with respect to e. Hence, an arbitrarily large 
blocklength is required near the threshold so that the above 
approximation formula is expected to be accurate. On the 
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other hand, the convergence is uniform for e 6 [0, eep] 
if the limit lirrin^oo Pb{n, e, oo) is continuous at eep as a 
function of e (i.e., lirrin^oo Pb{n, £bp, oo) = 0). In such 
cases, however, the coefficient of -nT 1 in ([TJ diverges as e 
approaches the threshold eep from below, since the threshold 
is given as ebp = {^'(fyp'fi))" 1 - Hence, an arbitrarily 
large blocklength is again required near the threshold so that 
the above-mentioned approximation formula is expected to 
be accurate. From the above facts, the approximation (fl~|i is 
accurate only for a small-e region which is often called an 
error floor. 

As an alternative approach, a method that is based on 
scaling law has been proposed ifTUl . ifTJl . which requires only 
a constant cost and is useful for estimation of the bit and 
block error probabilities near the threshold where the error 
probabilities behave like what is called a waterfall curve. This 
analysis permits finite-length optimization which maximizes 
rate of a code under a given blocklength, erasure probability 
and allowable error probability. 

Both of these two methods are, however, applicable only for 
infinite number of iterations, whereas the number of iterations 
is often constrained in practical applications due to limitation 
of resources, e.g., time, energy, etc., so that results for finite 
numbers of iterations should be more significant than those 
for infinite number of iterations. We therefore focus in this 
paper on an asymptotic bit error probability with respect to 
the blocklength when the number t of iterations is finite and 
fixed. The basic idea underlying our approach is to consider 
a large-n asymptotic expansion of the bit error probability 
and to evaluate the second dominant term in the asymptotic 
expansion. There exists a coefficient a(e, t) of nT 1 on the basis 
of which the asymptotic expansion of Pb(«, e, t) is expressed 
as 

P h (n,e,t) = P b (oo, e,t) + a(e,t)- + o ( -) . (2) 

n \n J 

The second term a(e,t)/n in the right-hand side of (O is 
determined by tree and single-cycle structures of local graphs, 
while the first term Pb(oo,e,i) is due to only tree local 
graphs. An important consequence of considering a finite- 
t asymptotic expansion is that the approximation formula 
derived by ignoring the term o (n^ 1 ) in <J2J is expected to be 
accurate for all e uniformly if the blocklength is sufficiently 
large, since the convergence lim„^oo Pb(n, e, t) is uniform for 
e G [0, 1], as we will see in later sections. Our main result is 
to derive a set of recursive formulas which allows evaluation 
of the coefficient a(e, t) for irregular ensembles. 

In Section [II] we define random ensembles of graphs used 
in this paper. In Subsection IIII-AI we see how the coefficient 
a(e, t) is decomposed into two components, one represent- 
ing contributions of cycle-free neighborhood graphs and the 
other representing contributions of single-cycle neighborhood 
graphs. In Subsection IIII-BI we obtain the component for 
cycle-free neighborhood graphs in a(e,t) by developing a 
generating function method. In Subsection IIII-CI we see how 
to enumerate the coefficient of n~ x in asymptotic expansion 
of the probability for single-cycle neighborhood graphs. The 
technique developed in Subsection IIII-CI is then used in the 
calculation of the contribution of single-cycle neighborhood 



graphs in Subsection IIII-EI via the single-cycle neighborhood 
graph ensemble defined in Subsection IIII-DI In Section |IV] 
we study the limit lim^oo a(e, t) for regular ensembles. In 
Section|V] we show that when the number of iterations is fixed, 
the large blocklength convergence is uniform with respect 
to e. It implies that, for sufficiently large blocklength, the 
approximation (01 is uniformly accurate for all e. Furthermore, 
in Section |VI] it is confirmed via numerical simulations that 
the approximations for several ensembles are accurate even 
for small blocklength. Finally, we conclude this paper in 
Section [Vll] The results of this paper have also been presented 
in conference papers lfl4l . IH31 . fl6l . 

II. Preliminaries 

A. Tanner graphs 

A Tanner graph G = (V,C,S\>,Sc,£) is a bipartite graph 
which is represented by a set V of variable nodes, a set C of 
check nodes, a set S\> of variable-node sockets, a set Sc of 
check-node sockets, and a set £ of edges connecting variable- 
node and check-node sockets. A node m is identified as a tuple 
of sockets associated with rn. To be precise, 

oo 

V c (J {(si, . . . , sfe) | si e <S V , . . . , s fc e S v } 

fc=i 

oo 

C C |J {(ii,..., ifc) I h e S e ,...,t k 6 S c }. 
fc=i 

Sockets associated with the same node are all distinct. Each 
variable-node socket s £ Sy is associated with one and only 
one node in V. Similarly, each check-node socket t e Sc 
is associated with one and only one node in C. An edge e 
is identified as a pair of sockets which connect to e. To be 
precise, 

SC {(s,t) | s€S v ,teS c }. 

Each socket connects to one and only one edge. The number 
of variable-node sockets, the number of check-node sockets 
and the number of edges are equal, i.e., |<Sy| = \Sc\ = \£\. 

B. Irregular LDPC code ensembles 

An (n, L{x) , R{x))-irregular ensemble is a random ensem- 
ble of LDPC codes of blocklength n which are represented 
by Tanner graphs with variable-node degree distribution poly- 
nomial L{x) from node perspective and check-node degree 
distribution polynomial R(x) from node perspective [ 1|. These 
two degree distribution polynomials are expressed as 

L(x) :^^L lX \ R{x) := ^ RjX j . 

i i 

Each Tanner graph in the (n, L(x), i?(a;))-irregular ensemble 
has n variable nodes, fraction Li of variable nodes of degree i, 
and fraction Rj of check nodes of degree j. The sets V, C, S\> 
and Sc defining nodes and sockets in the Tanner graphs are 
arbitrarily fixed in an ensemble. Each instance of the edge set 
£ is chosen randomly from all El possible realizations with 
uniform probability, where E := \£\ = nL'(l) is the number 
of edges of the Tanner graphs. 
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We also use degree distribution polynomials X(x) and p[x) 
from edge perspective, which are defined as 

L'(x) 



X(x) 



p(x) 
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~ L'{1) 
R\x) 

■ R'(iy 



An (n, L(x), i?(ir))-irregular ensemble has fraction At of 
edges incident to a variable node of degree i and fraction 
/}j of edges incident to a check node of degree j. An 
(n, i?(.x))-irregular ensemble is also referred to as an 
(n, X(x), ( o(x))-irregular ensemble. 

The bit error probability of (n, X(x), p(x))-irregular ensem- 
ble is defined as the average bit error probability of instance 
codes. In this paper, we deal with an asymptotic bit error 
probability with respect to blocklength with the fixed degree 
distributions (X(x), p(x)). In the following, we will also use 
the notation (X(x), p(x)) to specify an irregular ensemble 
when the blocklength is not relevant. 

C. Neighborhood graph ensembles 

Distance of two nodes in a Tanner graph is defined as the 
number of check nodes in the shortest path between the two 
nodes except both ends. A neighborhood graph of depth t 
of a variable node vq is a subgraph which consists of the 
variable nodes and the check nodes with distance from vq not 
greater than t and (t — 1), respectively. Each neighborhood 
graph is expressed as G = (Vm,Cm,vq,Sv n ,S Cm ,£n)- Sets 
VjV> C_s/, Sv^, Sex and £j\f are a set of variable nodes, a set 
of check nodes, a set of variable-node sockets, a set of check- 
node sockets and a set of edges, respectively. Expressions and 
roles of W, Cjv", Sy H , and £jV are the same as those of 
V, C, <5>v, Sc and £ for a Tanner graph, respectively. A variable 
node vq e W is called the root node. Depth of a node in G 
is the distance from the root node vq. Variable nodes of depth 
t may have sockets which do not connect to any edges. With 
an abuse of notations, we will also write V(G) := VV and 
C{G) := C M . 

A neighborhood graph ensemble Aft{n, X(x), p(x)) in- 
duced by an (n, X(x), p(x))-irregular ensemble is an en- 
semble of neighborhood graphs of depth t. Each neigh- 
borhood graph G is associated with the probability P„(G) 
which is defined by the following steps. We first con- 
sider a Tanner graph (V, C, <Sy, Sc, £ ) generated from the 
(n, X(x), p(x))-irregular ensemble. For a neighborhood graph 
G = (Vjsr,Cjy,Vo,Sv„,Sc„,£jv) with W C V, Cm C C, 
Svm Q <5v, Sq n C Sc and Sjy C £, where the root node 
vq is chosen uniformly from V, and where a variable node 
»gV and a check node c G C are members of W and Cw if 
and only if their distances from vq are not greater than t and 
(t— 1), respectively. Similarly, a variable-node socket s £ <Sy 
and a check-node socket t € Sc are members of Sy^ and 
Sc M if and only if s and £ are associated with nodes in VV 
and Cm, respectively. An edge (s,t) e £ is a member of £jj 
if and only if s S 5^ and i € ^c^- 

The random choice of the edge set £ in the original irregular 
ensemble induces a probability distribution over the set of 



neighborhood graphs, under which each possible neighbor- 
hood graph G has a probability 



Qn(G) 



nE{E-l) — {E-(k-l)) ' 
0, 



if E > k 

otherwise 



where E = nL'(l) is the number of edges in the whole 
Tanner graph, as defined in Subsection IH-BI and where 
k denotes the number of edges in G. For convenience, 
we will use a marginalized probability P„(-) which is 
induced from Q„(-) via the equivalence relation defined 
as follows: G = (Vjv, Cj^, vo, Sv^-, Sc u , £jv) and G' = 
(Vj\r' \Cj*f' ,v' ,Sy ,Sc ,£jy') are equivalent if and only if 
there exist bijections cry : <Sy^ — > S\>^. and ac ■ Sc^ Sc' N 
such that 

cl. (a v (s),a c (t)) G £m for all (s,t) £ £jy 
c2. CTy(u ) = v' 

c3. W £ Va^\wo, Bv' € VatVo s - l a v(v) = v' 

c4. Vc e Cm, 3c' G <V s.t. a c (c) = c! 
where cry(w) (respectively <jc(c)) are tuples whose i-th ele- 
ment is the image of i-th element of v (respectively c) under 
cry (respectively ac) and where m = m' if and only if they are 
equal under cyclic shift for tuples m and m' of sockets. This 
equivalence relation is weaker than what is used in Q„( ) and 
stronger than the conventional equivalence relation in graph 
theory which does not distinguish sockets. 

Under this equivalence relation, the number of neighbor- 
hood graphs equivalent to G is 



nL ]vo] Yl i I ( nL * 



1=0 



;=o 




J] } [ {mRi - l)j 



1=0 



n n ( E Pi - w 



1=0 



where Vi denotes the number of variable nodes of degree i 
in G, where Cj denotes the number of check nodes of degree 
j in G, and where m denotes the number of check nodes in 
the whole Tanner graph, i.e., m = nL'(l)/R'(l). Hence, the 
probability P n (-) which marginalizes equivalent neighborhood 
graphs is given as 



P«(G) = L 



a* ufoHEx, - u) n n^o 1 ^ - m 



(3) 



This defines the probability associated with a neighborhood 
graph G in the neighborhood graph ensemble. 

Since E = 8(n), the denominator and the numerator 
are Q(n k ) and Q(n w ), respectively, where w denotes the 
number of nodes in G except the root node. One therefore 
has P„(G) = Q(n w ~ k ). Since the number of cycles in G is 
(k — w), the next lemma follows. 

Lemma 1. For a neighborhood graph G which has c cycles, 

P„(G) = Q(n- C ). 

This lemma plays a key role in this paper. Classification of 
neighborhood graphs according to the number of cycles is 
also considered in j4j. 
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D. Tree ensembles 

From Lemma Q] neighborhood graphs of a fixed depth with 
cycles are not generated in the large-blocklength limit. To be 
precise, 



P QO (G) := lim P„(G) = L 



-I n 

veV(G)\v 



A i 



«i n p \c\ 

cec(G) 



(4) 

for a tree graph G and P oc (G) = for any graph G 
with cycles. The ensemble of tree neighborhood graphs with 
probability P 00(G) is called the tree ensemble from node 
perspective, and is denoted by %(X(x), p{x)). 

We also define two other tree neighborhood graph ensem- 
bles, namely tree neighborhood graph ensembles from edge 
perspective , Tf(X(x), p(x)) and T t c {X(x), p(x)). Neighbor- 
hood graphs in T i v (X(x), p(x)) and T t c (X(x) , p(x)) are rooted 
at an edge incident to a variable node and a check node, 
respectively. The number of check nodes in the shortest path 
from the node connected to the root edge to any node is not 
greater than t. Only variable nodes which have distance t from 
the node connected to the root edge have sockets which do 
not connect to any edges. The probability of a neighborhood 
graph G rooted at an edge in both ensembles is 

n a m n pm- 

vGV(G) ceC(G) 

The ensembles 7t(X(x) , p(x)) and T t v (X(x) , p(x)) are also 
defined in ifTTl . 



III. Main result 
A. The decomposition of the coefficient of nT 1 

For each variable node, an error occurrence after t BP iter- 
ations depends only on a realization of a neighborhood graph 
G of depth t and realizations of channel outputs corresponding 
to variable nodes in G. In other words, the bit error probability 
of irregular ensemble is 



Pb(n,e,i) 



GGSt 



,(G)P b (e,G) 



(5) 



where Q t denotes the set of all neighborhood graphs of depth 
f, and where P b (e, G) denotes the error probability of the root 
node of G after t iterations when the erasure probability of 
each node in G is initialized with e. From Lemma Q] it holds 
that 



P b (oo, e,t) 



GeT t 



3 (G)P b (e,G) 



where % denotes the set of all cycle-free neighborhood graphs. 
This fact allows us to calculate the limit of the bit error 
probability P b (oo,e, t) := lirrtn-^oo Pb(w, e, t) in a recursive 
manner, leading to the idea of density evolution. 

Lemma 2 (Density evolution @). Let Q e (t) denote erasure 
probability of messages into check nodes at t-th iteration, and 
let P e (t) denote erasure probability of messages into variable 



nodes at t-th iteration in the limit of infinite blocklength. Then 

P h {oo,e,t) = eL{P £ (t)) 

Q e {t) = eX(P e {t-l)) 

'l, ift = 



PS) 



1 — p(\ — Q e (t)), otherwise. 



On the other hand, one observes from Lemma Q] that the 
second and the third dominant terms are 8(n _1 ) and 6(n -2 ), 
respectively. In other words, one has the following large-n 
asymptotic expansion of Pb(n, e, t): 

Pb(n,e,f) =Pb(oo,e,t)+a(e,t)- + e [\ 

n \n z 

where the coefficient a(e,t) of n _1 is defined as 

a(e,t) := lim n(Pb(n, e, t) — Pb(oo, e, t)). 

n— foo 

Moreover, Lemma Q] tells us that a(e, t) can be decomposed 
into two components as follows: 

a(e, t)= lim n [ V P„(G)P b (e, G) - P b (oo, e, t) j 
\GeTt I 

+ lim n Y P„(G)P b (e,G) 



Ges t 



£,t)+7M) 



where St denotes the set of all single-cycle neighborhood 
graphs and where the components (3(e,t) and j(e, t) rep- 
resent contributions of cycle-free and single-cycle neighbor- 
hood graphs, respectively. In Subsection IIII-BI and Subsec- 
tion UlLEl recursive formulas to evaluate j3(e,t) and j(e,t) 
for (X(x), p(a;))-irregular ensembles are derived, respectively. 

B. The contribution of cycle-free neighborhood graphs 

The component /3(e,t) for cycle-free neighborhood graphs 
is calculated as 

P(e, t) := lim n ( V P„(G)P b (e, G) - P b (oo, e, *) ) 
\GeT t / 

= V [lim n(P n (G)-P 00 (G))lp b ( e ,G). 

* — » Ln— >oo 

Ger t 

From (O and (O, the contributions of a cycle-free neigh- 
borhood graph G to /3(e, t) is obtained as 

L M II A M IT Pk|Pb(e,G) 

wEV(Gf)\«o cGC(G) 



lim n 

n— >ao 



Pi 



- 1 



L M J] A|„| Y[ P\c\ p b{e,G)^— 



v€V(G)\v ceC(G) 



(1) 



Hk - 1) - x; f - 1) - e i: c i^ - x ) 
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Hence, f3(e,t) is obtained via expectation, denoted by E f [-], 
on the tree ensemble 7~t(\(x), p(x)) of depth t from node 
perspective as 



1 



227(1) 



E t [K(K - 1)P] - V f E t [Vi(Vi - 1)P] 



(6) 



where X, Vi and Cj denote random variables representing the 
number of edges, the number of variable nodes of degree i, 
and the number of check nodes of degree j, respectively, and 
where P denotes the erasure probability of the root node after 
t BP iterations. 

The three expectations in (O are obtained using generating 
functions as 

d 2 E t [x K P] 



E t [K(K-l)P] = 
E t [Vi(Vi - 1)P] = 
E t [C j (C j -l)P] = 



dx 2 
d 2 E t [x v *P] 



x=l 



dx 2 
d 2 E t [x c =P] 



dx 2 



(7) 
(8) 
(9) 



In order to deal with these generating functions, we now define 
the following "canonical" generating function: 



$(t; {y k }, { Zl }) =E t 



iwn 



z f-p 



L k 



(10) 



The three generating functions that appear in the right-hand 
sides of ©-(O are obtained from {yk}, {zi}) as 

1 



„K 



P] = - *(*; {y k }, {zi})\ Vk=x , Zl=x 



E t [x v >P] 
E t [x c iP] 



S(t; {y k }, {zi})\ yi= 
3>(t; {y k }, { Zl })\ : 



for all k, I 



■j=x\ y k = 



*k; zi=l V/^j 



The key idea here is that one can evaluate the canonical 
generating function <E>(i; {yk}, {zi}) via extending density 
evolution in such a way that "densities" to be updated in 
density evolution incorporate the auxiliary variables {yk} and 
{zi}. We call our extension the augmented density evolution. 
In the conventional density evolution, expectation of density 
of messages over a tree ensemble is calculated in a recursive 
way. In the augmented density evolution, on the other hand, 
one considers, for each tree, a product of the density of 
messages at the root node and a monomial reflecting degree 
histogram of the tree, and calculate its expectation over the 
tree ensemble, which can be performed recursively in a similar 
way to density evolution. The canonical generating function 
$(£; {yk}, {zi}) in the general case is thus a polynomial 
whose coefficients are conical combinations of densities. Since 
we are assuming BEC(e), we only have to deal with erasure 
probabilities of messages instead of densities of messages, as 
shown in Lemma [2] Hence, the canonical generating function 
<£>(t; {yk}, {zi}) is obtained by a recursive calculation of 
polynomials in {yk} and {z{} with real-valued coefficients. 
The next lemma provides a set of recursive formulas to 
evaluate the canonical generating function {yk}, {zi}). 



Lemma 3. The canonical generating function 
}Vk}, {zi}) is given by 



where 



F(t) 



m {y k }, { Zl }) = e£(F(t)) 
fl, ift = 



\P(ffW) - P(G(t)), otherwise 
G{t) :=C{f{t-l))-eC{F{t-l)) 

/(*):= I'' ift = ° 

\V(g(t)), otherwise 

g(t) := C(f(t - 1)) 



and where 



C(x) :=]>> 

i 

V{x) :=X> 



3-1 



Proof: The generating function is calculated as 



E, 



L k 

= E T - 



= e£ E; 



& i 



L k 



where m denotes a random variable corresponding to the 
degree of the root node, where E m denotes expectation with 
respect to the degree of the root node, where E£[-] denotes 
expectation on T t c , and where Q denotes a random variable 
corresponding to erasure probability of messages transmitted 
to the root edge at t-th iteration. Now define 



f(t)=E<; 



m 

F(t) 

G(t) = EU 



Et 



i 

IK* IK' 

k I 

ikikq 

I 

IKIKa-^) 



where E"[-] denotes expectation on T t v - The functions f(t) 
and g(t) are the generating functions of {Vk} and {C\} on 
the ensembles T t c and 7fL\, respectively. The functions F(t) 
and G(t) are reweighted versions of the generating functions, 
where reweighting is done on the basis of erasure probability 
at the root node. It should be noted that dependence of 
these functions on the auxiliary variables {yk} and {zi} is 
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implicit in the notation. The desired expectations are calculated 
recursively as 

/(0) = P(0) = 1 

fit) = E c t [^(t)™- 1 ] = V(s{t)), if t > 1 

«?(*) = EJLi [y rn f(t - I)"- 1 ] = £(/(« - 1)) 

F{t) = Ej [z m (.g(t)™- 1 - G(t)'"- 1 )] 

= f(t)-V(G(t)), if * > 1 

G(t) = E"^ [y m (/(* - I)"- 1 - eP(i - 

- ff (t) - e£(P(f - 1)). 

■ 

Considering appropriate derivatives of the recursive for- 
mulas given by Lemma [3] one obtains explicit formulas to 
evaluate the three expectations in (O recursively, on the basis 
of which one can evaluate /3(e, i) explicitly. The derivation is 
elaborate but straightforward, so that we omit details of the 
derivation and only show the end result. Let us define, for 
n = 1 and 2, 

d n f(t)\y k =x,z,=x for all k,l 



/(")(*) 

/i n) M 



9 n /(i) 



yfc — 1, zi — 1 for all fc,Z 



y k — 1 , — 1 for all k.l 

Similar definitions are applied to git), Fit) and G(t) to 
define g^ n \t), gi n \t,i), etc. The resulting 24 functions are 
to be used to evaluate the relevant expectations, and the 
recursive formulas of these functions used in the evaluation 
are summarized in the next theorem. 

Theorem 1. /3(e, t) for (A(x), pix))-irregular ensembles is 
calculated as 



/8(c,t) = 



1 



2L'(1) 



E t [KiK-l)P] 



£ t-E*[V5(VS - 1)P] - J] - 1)P] 

Pi 



where E t [K{K-l)P\ E t [F;(V; - 1)P] and E t [Cj(Cj - 1)P] 
are calculated by Q, ([5]) ant/ (O, respectively. The functions 
Peit) and Q e it) appearing in these formulas are to be 
evaluated recursively via the conventional density evolution 
(Lemma 2). 



0, ift = 

[1 + otherwise 

<?«(*) = l + A'(l)/«(t-l) 

'0, ift = Q 

fW{t)-p(l-Q e {t)) 

-p'il-Q f Xt))G {1 \t), otherwise 

= <?«(*) -eA(P e (*-l)) 
-eA'(P e (t-l))P«(i-l) 



P«(i) 



f i2) it) 
9 {2) it) 

F^it) 
G^ 2 \t) 



{0, ift = 

2p'(l).9 (1) W 
+p"il)g {1 \t) 2 +p'il)g ( - 2 \t), otherwise 

2A'(1)/ W (i - 1) + \"(l)fW(t - l) 2 
+ A'(l)/( 2 )(t-l) 
'0, ift = 

fW(t)-2f/(l-Q&))GM(t) 
- P "il~Q € it))G^it) 2 
-p'(l-Q e (t))G< 2 )(i), otherwise 
g( 2 \t)-2e\'iPit-l))F (1 \t-l) 
-e\"(P e (t- l))P (1) (i- l) 2 
-eA'(P e (t-l))P^(t-l) 



Ej [if (ii" - 1)P] = eL"(P e (t))F (1) (i) 2 + eL'iP e it))F^it) 

(7) 



pi x) M 



JO, (ft = 

1 //(l)</£^(t, i), otherwise 

\'{l)fW(t-l,i) + Xi 

jo, ift = 

\fv 1} it,i) - p'il - Qzit))G ( v ] it,i), otherwise 
g^it,i)-e\'iP e it-l))F^it-l,i) 

- ex^t -if- 1 



/i 2) (M) 



o, ift = a 

yi^gi^it^i) 2 + p'il)gi 2 \t,i), otherwise 
gi 2 \t, i) = A"(l)/W(i - 1, i) 2 + X'(l)fi 2 \t - 1, i) 
+ 2A i (i-l)/W(t-l,<) 

fa ift = o 

Fi 2 \t,i) = J /i 2 )(i,i)-p"(l-Q e (i))GW(t,0 2 

[ -p'(l - Q e (i))Gi 2) (M), otherwise 

G (2 \t,t) = tf\t : i) - eX"iPit ~ l))F«(i - l,z) 2 
-eA'(P e (t-l))Pi 2 )(t-l,i) 
-2eXiii-l)P e (t-iy- 2 Fy\t-l,i) 

E t [ViiVi - 1)P] = eL"iPS))F^it,if 

+ €L'iP,it))Fl 2) it,i) + 2€L l iP c it)^ 1 F^it,i) (8) 

/(i)( t)i ) = I 0, x ^ = 

1 /9 / (l)(7c 1 ' ) (i, i) + Pj, otherwise 

gPit,j) = X'(l)fPit-l,j) 

{a ift = o 

fP(t,j)-(/(l-Q e (t))GP(t,j) 
—pjil — Q.it)) 3 , otherwise 

G^itJ) = j) - eA'(P e (* - l))P«(i - l,j) 
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Si = 1, S 2 = 

Type I 




s = 5 
Type VI 



Fig. 1. Six types of single-cycle neighborhood graphs. All nodes which are not included in the two minimum path from the root node to the deepest node 
in the cycle are not described in the above figure. These are classified according to whether the shallowest and the deepest nodes in the cycle are variable 
nodes, check nodes or the root node. A depth of the shallowest node in the cycle corresponds to ax- The number of nodes in the shortest path from the root 
node to the deepest node in the cycle corresponds to S2 + 1 and s + 1. 



/i 2) (U) 



9 { c 2) (t,j) = 



0, ift = 

p"(lU 1) (t,j) 2 +p'(l)gi 2 \t,j) 

+2pj(j — l)g[ \ (t, j), otherwise 
A"(l)/«(t-l,i) 2 + A'(l)/i 2 )(t-l,i) 
0, ift = 







root node to the deepest node in the cycle. We classify single- 
cycle neighborhood graphs into six types of subgraphs S(G) as 
shown in Fig. [TJ They are classified according to whether the 
shallowest node in the cycle is a non-root variable, a check, or 
the root node, as well as whether the deepest node in the cycle 
is a variable or check node. Types I to IV of neighborhood 
graphs have two parameters: si corresponding to the depth 
of the shallowest node in the cycle, and s 2 for which S2 + 1 
equals to the number of nodes in the shortest path from the 
root node to the deepest node in the cycle. Types V and VI 
of neighborhood graphs have a parameter s which plays the 
same role as s 2 in Types I to IV. The set of single-cycle 
neighborhood graphs of Type I and Type II with the parameters 
Si and s 2 is denoted by S v (t, si, s 2 ). The sets S c (t, si, s 2 ) and 
S r (t, s) are defined in the similar way. 

We consider marginalization of the probability using the 
^ t [Cj{Cj- 1)P] = eL"(P e (t))F^ 1) (t,j) 2 +eL'(P e (t))F^ ) (t,j). classification of neighborhood graphs. The probability P„(G) 

(9) of a single-cycle neighborhood graph G is 



fP(t,j)-p"(l-Q e (t))G^(t,jf 
-p>(l-Q e (t))G?\t,j) 
-2 Pj (j - 1)(1 - Q c (t)y~ 2 



otherwise 



-e\'(P e (t 



e\"(P £ (t-l))F^(t 



l,i) 2 



C. Method of enumeration 

In order to calculate the coefficient a(e,t) of n _1 , it is 
necessary to evaluate the contribution of single-cycle neigh- 
borhood graphs, i.e., 

7 (c,t) := lim n V P„(G)P b (e, G). 

n— >oo ^ — ' 

GeSt 

For ease of the explanation of how to evaluate j(e,t), which 
is deferred to Subsection IIII-EI we consider in this subsection 
a different quantity, namely the coefficient of n^ 1 in the 
probability of single-cycle neighborhood graphs: 

£(t) := lim n Y] P„(G). 
GeSt 

Methods for enumeration of £(t) introduced in this subsec- 
tion will be extended to those for calculation of ~f(e,t) in 
Subsection IIII-EI In both calculations, we consider subgraph 
S(G) of a single-cycle neighborhood graph G consisting of 
nodes which are included by the two shortest paths from the 



L 



Ui ngo^g - it) Uj ut \pjE - m 
n<=o ( E - 1 > 



Since E = nL'(l), we obtain the coefficient of n 1 as 

Jim nF n (G) = j±-L ]vol J] A,^ J] p ]c] . 

v ' veV(G) cec(G) 

In order to enumerate the coefficient of n^ 1 in the proba- 
bility of single-cycle neighborhood graphs, we consider an 
equivalence relation in which positions of sockets connected 
to a socket associated with a node in S(G) are not distin- 
guished, which is weaker than what is used in P n (-). The 
sets of representatives of the resulting equivalence classes 
in S v (t, Si, S2), S c (t,si,S2) and S r (t,s) are denoted by 
S v (t, Si, S2), S c (t, Si, s 2 ) and S r (t, s), respectively. The coef- 
ficients of n^ 1 in the probability of single-cycle neighborhood 
graphs of Type I and Type II with parameters Si and s 2 are 
evaluated in a unified way (s 2 is even for Type I and odd for 
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Type II), and are obtained as 



lim 

n— >oo 



J2 nF "(G) 



GeS„(t,si,s 2 ) 

l \v \\v \ n 

■\w\-i 



L'(l) 



E 



Ge5„(t,8i,» 2 ) 

*m(M-i) II Pici(kl-i) 



cec(S(G)) 



x A 



n 



A: 



n 



»eV(G)\V(S(G)) 



ceC(G)\C(S(G)) 



1 



= iA"(l)p'(l) 2 (A'(l)p'(l))^— 2 



(10) 



where it) denotes the shallowest variable node in the cycle. In 
the first equality in ( [Tol l, single-cycle neighborhood graphs 
of Type I or Type II are marginalized according to the 
equivalence relation. In the second equality, by the marginal- 
izations, quantities corresponding to nodes not included in 
S(G) become 1, and quantities corresponding to the root node, 
the shallowest node in the cycle, other variable nodes in S(G), 
and check nodes in S(G) become L'(l), A"(l)/2, A'(l) and 
p'(l), respectively. The concept of the equivalence classes 
S v (t, Si, S2), S c (t, si, S2) and S r (t,s) is useful not only for 
the calculation (TTOb but also for the calculation of j(e,t) in 
Subsection IIII-EI 

In the same way, the coefficients of n^ 1 in the probability 
of single-cycle neighborhood graphs of Type III and Type IV 
with parameters si and S2 are calculated as 

ip"(l)A'(l)(A'(iy(l)r- s - 2 

and those for Type V and Type VI with the parameter s are 
calculated as 

Similar calculations are also used in |]4). The classification of 
single-cycle neighborhood graphs in this subsection is finer 
than that in |4) for the purpose of calculation of j(e,t) in 
Subsection IIII-EI Summing up the above contributions of all 
types of single-cycle neighborhood graphs, we obtain 



t-i 



2/ 



\s 2 — Sl— 2 



= E E iA"(iy(i) 2 (V(i)p'(i)) s 

Sl=l S2=2Sl+l 

t-1 2t 

^s 2 -;si— 2 

^ j 1 ] 1 /\ < \ 11 -A ! r 1 1 i \ 1 1 ■ 



si=0 s 2 =2si+2 



E E ^"(i)A'(i)(A'(iy(i)) 

+2 

2t 

A"(1)P'(1) ' 



^l-CA'aya^-^a-CA'ayci))*) 



/(l)A'(l) 



(l-A'(l)p'(l)) 2 

(i-A'(iy(i)) 2 

1 - (A'(l)p'(l)) 2 * 



A'(1)P'(1)- 



1-A'(l)p'(l) 



It should be noted that the above result can alternatively 
be obtained via the generating function method described in 
the previous subsection. Indeed, since the probability of all 



neighborhood graphs is exactly 1 and since the probability of 
neighborhood graphs which contain more than one cycle is 
<d(n~ 2 ), the coefficient of n^ 1 in the probability of cycle- 
free neighborhood graphs is — £(t), i.e., the probability of 
tree neighborhood graphs is 1 — £(t)/n+ 6(n -2 ). Hence, the 
above result for the quantity £(t) is obtained by enumeration 
of the coefficient of nT 1 in the probability of cycle-free 
neighborhood graphs — (3(1, t) using the generating function 
method in the previous subsection. 

D. Single-cycle neighborhood graph ensembles 

Single cycle neighborhood graph ensembles are defined 
in this subsection in order to make the description of the 
calculation of j{e,t) in the next subsection more tractable. 
A single-cycle neighborhood graph ensemble for an arbitrary 
fixed type and parameters is defined not in terms of single- 
cycle neighborhood graphs but in terms of representatives 
of their equivalence classes, with the specified type and 
parameters. The definition of single-cycle neighborhood graph 
ensembles is motivated by (TTOb . The probability, to be defined 
in this subsection, of a representative, denoted as G by a slight 
abuse of notation, can be considered as the large blocklength 
limit of the conditional probability, measured by the neigh- 
borhood graph ensemble, of the single-cycle neighborhood 
graphs in the equivalence class represented by G conditioned 
on that a single-cycle neighborhood graph has a particular 
type and parameters. The probability of a representative G of 
an equivalence class of single-cycle neighborhood graphs in 
S v (t, si,s 2 ) is 

p(t,«i,«2)m = ^i""! 1 ^ 1 a h(H ~ iXIH -2) 

1 ' ' L'(l) A"(l) 



n 



Am(M-I) -q P| c |(|c|-1) 



A'(l) 11 p'(l) 



n 



A, 



n p\o\ (U) 



u£V(G)\V(S(G)) c£C(G)\C(S(G)) 

where w denotes the shallowest variable node in the cycle. 
Similarly, the probability of a representative G of an equiva- 
lence class of single-cycle neighborhood graphs in S c (t, s%, S2) 
is 

rrn •= L ^ v °\ p ^ s \ ~ Z 2) 

[ >' L'(l) p"(l) 

n yj - jj tt pici(i c i - jj 
A'(l) 

veV(S(G))\v y ' ceC(S(G))\s r v ; 

x n a i-i n p\°\ 

v£V(G)\V{S{G)) c£C(G)\C{S(G)) 

where s denotes the shallowest check node in the cycle, and 
the probability of G € S r (t, s) is 

P p )(G):= ^NiN(NI-i) n Aw 



veV(G)\V(S(G)) 



nrr A| V |(|u|-i) p| C |(|c|-i) 
p ^ n m n W) ■ 

c€C(G)\C(S(G)) v<ES(G)\v y ' c£S(G) P V ; 
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These ensembles are used in Subsection IIII-EI for calculation Theorem 2. 7(5, t) for (A(x) , p(x))-irregular ensembles is 
of 7(6, t). calculated as 



E. The contribution of single-cycle neighborhood graphs 



t-i 2t 

7( e ,*)=E E Fv(t,S!,S 2 ) 
si=l s 2 =2si + l 



The contribution 7(e,t) of single-cycle neighborhood * _1 2t 2t 

graphs can be decomposed according to the types and pa- + ^cft, si } S2) + F r s ) 

rameters of single-cycle neighborhood graphs. si=0s 2 =2si+2 s=i 

x „ where F v (t, si, S2), F c (t, Si,S2) F r (t,s) are shown in 

7(e, *) = 2-» („ lim o " P «( G )) P b(e J G) (El, (O and 01, respectively. IfX"(l) = 0, F,(t, Si,s 2 ) u 



Ge5t defined as 0. 

t-l 2t 



V" V" V" ( lim nP„(G))P b (e, G) A derivation of F v (t,s\,S2) is described in the following. 

ii=ia a =2ai+iGfeS v (t,ai,aa) ™^°° Similarly to ([Toll, the contribution F„(t,si,s 2 ) of neighbor- 

t _l 2t hood graphs in S v (t, si, S2) to 7(e, t) is obtained as 

EE E fcnF„( G ))P b(e ,G) Fv{t )= E ( limnPn(G) ) Pb(e , G) 



si=0a 3 =2si+2GteS e (t,»i,» 3 ) v ' Vn->-oo 

2t GeS„(Mi,s2) 

^E E ( n limnP n (G))P b ( e) G) = * £ 

x; f; F„(t, sl , S2 ) x n A M d«i - 1) n p, C |(| C |-i) 



81 = 1 S 2 =2si+1 

t-l 2t 2t 



E E ^, S i, S2 )+E^ s ) H v 2 



v£V(S(G))\{v ,w} c£C(S(G)) 
XA I-|(' W| 9 j II A M II 



si=0 s 2 =2si+2 



u€V(G)\V(S(G)) ceC(G)\C(S(G)) 



xP b (e,G) 



where F„(i, S1.S2), F c (i, si,s 2 ) and F r (t,s) are the contri- 1 „,,„,,, „ 

= — A (Do (1) (X (Do '(D~) S2 ~ Sl ~ 
butions of single-cycle neighborhood graphs in S v (t, s%, S2), 2 

5 c (i, si, s 2 ) and S r (i, s), respectively. A set of formulas for „ L|„ ||« | tt -"V|(M _ 1) 

calculations of these quantities are shown in the next theorem. x 2-^ L/JT) X'(l) 

Ge<s„(i,ai,a 2 ) 1 ; vev(s(G))\{v ,w} K ' 

Je, if t = _ jp, if s = 

/(M ' P):= V^f.9(M-l, P ), otherwise' 3(< ' S ' P) != \l - ^f^(l - /(* - 1, s,p)), otherwise 

1, ifs = 
G l{ t, s) :=\(l- ^f 1 ) 2 + 2^(m (x _ £^M) /(t _ x, s , l) 

+ ( p ' (1 ~g (t)) ) 2 G 2 ft- otherwise 

, _ . 6 v(i) . il s-0 

si ^ £ ymmy Gl{t ia _ 1}j otherwiS e 

fi A'(P e (t)) ifs-0 
G ft s) •= J A ' (1) ' 2 ~~ 

3 ' S 1 1 - 2f(t, s + 1, 1) + (e V %* )? ) Gi(*,«-1), otherwise 

F„(t, 8l , s 2 ) - iA"(l)p'(l) 2 (A'(iy (I))'-— 2 Q £ (t + 1) 

X g \t, 81 - 1, 1 II - e Gi(t - s x , s 2 - 2s x - 1) 1 1 (12) 

Fc(*, *i, *a) = ip"(l)A'(l)(A'(l)p'(l))^- sl - 2 Q £ (t + 1) 5 (t, s lt 1 - P " {1 ~ ® e( f - Sl)) G 3 (t - Sl -l,s 2 ~ 2s, - 2) 

(13) 

F r (t,s) = ^^(lyCl))^^^^^^-!) (14) 
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X 



n 



P| c |(|c|-1)\ A N (H-l)(N-2) 



A"(l) 



\c£C(S(G)) 

H X ]v \ [] P| c |P b (e,G) 



uGV(G)\V(S(G)) c€C(G)\C(S(G)) 



1 



A"(iy(i) 2 (A'(iy(i)r— 2 



x £ p(*'^^)(G)P b (e,G) 
Ges„(t,«i,» 2 ) 



Hence, we have to calculate the expected error probability 
over a single-cycle neighborhood ensemble. Marginalizing the 
non-cycle part of S(G) and trees incident to them, if any, we 
obtain 

F v (t, Sl ,s 2 ) = ^X"(l)p'(l) 2 (X'(l)p'(l)r-^- 2 

TT \v\(\v\ - !) TT P\c\(\c\ - 1) 

X ^ H A'(l) 11 p'(l) 

Y,Z veV(Y)\w V ; cGC(Y) PV ' 

A H (H-l)(H-2) TT , TT 

X ^ A"(l) U Am P|c| 



x e 



«6V(Z) ceC(Z) 

L'{P<(t)) ( 1 P'(I-QS)) / c X'(P e (t-l)) 



L'(l) 
(1-P) 



P'(l) 



P'(l) 



(15) 



where F denotes the subgraph which consists of nodes in 
the cycle, where Z denotes trees incident to Y, and where 
p denotes erasure probability of the message from w to the 
shallow check node connected to w. The calculation of a 
non-cycle part in (15[ is similar to the derivation of density 
evolution in Lemma [2] Equation dl3T > is calculated as 

±x"(i)p'(i) 2 (x'(i)p'(i)y>-^- 2 Q e (t + i) 

\ (. 1 1 P '(l-Qe(t-S 1+ 1)) 

x g t, si - 1,1 — I 1 



TT \v\{\v\ ~ 1) TT PjcKN - 1) 

^ 11 A'(i) 11 p'h) 

Y.ZveV(Y)\w K ' cGC(Y) FV ; 

x — A^m 11 A| " 1 11 PlclP 

v ' vev(z) cec(z) 

Hence, if one can prove the equality 

TT \v\(\v\ - 1) TT P|c|(H ~ 1) 

^ 11 A'(l) 11 p'(l) 

Y,ZveV(Y)\w y ' ceC(Y) PV ; 

X\ w \{\w\ - l){\w\ - 2) 

X — rTTT^ {} A|„| }} /9| C |P 



A"(l) 



u£V(Z) ceC(Z) 



= e X " (P ^ {1) Sl)) Gi(t - si, s 2 - 28i - 1) (16) 

then (TTZt will immediately be obtained. 

Now we prove ([TBT l. First, marginalizing u> and trees in- 
cident to w, denoted by Z w , the left-hand side of (fTST l is 



calculated as 

e n 



X\y\{\v\ - 1) 
A'(l) 



n 

c£C(Y\w) 



P | C |(|C| - 1) 



P'(l) 



n 



A 



v\ 



n 



v£_V(Z\Z w ) cGC{Z\Z w ) 



X"(P e (t- Sl )) 
A"(l) 



q (17) 



where g denotes the probability that two messages into w from 
the check nodes connected to w in the cycle are both erased. 
Let ci and c 2 denote the check nodes in the cycle incident to 
w. If Ci and c 2 are the same, i.e., if s 2 — 2si — 1 = holds, 
then q = 1. Otherwise, q is decomposed to four components 
as 

q = P(ei,e 2 ) = P(Ai, A 2 , e x , e 2 ) + P{A\, A 2 , e u e 2 ) 
+ P(Ai, A 2 , ei, e 2 ) + P(Ai,A 2 ,ei,e 2 ) 
= P(Ai)P{A 2 ) + P(ei | Ai, A 2 )P(li)P(A 2 ) 
+ P(e 2 | Ai,A 2 )P{A 1 )P(A 2 ) + P(A 1 ,A 2 ,e 1 ,e 2 ) (18) 

where ei and e 2 denote the events that the messages from Ci 
and c 2 to w are erased, respectively, and where A\ and A 2 
denote the events that at least one message from outside the 
cycle into ci and c 2 is erased, respectively. Calculating the 
marginalization in (JTTJ, the first term in ( fT8l becomes 



1 - 



p'(l-Q e (t-ai)) 



(19) 



Each of the second and third terms becomes 



P'(l 



<-(t-si)) 



P'(l) 



p'(l - Q e (* - a x )) 
P'(l) 

X /(*- 81-1,82-28! -1,1). (20) 



At last, the fourth term becomes 



p'(l-Q e (t-si)) 



P'(l) 



(21) 



where r denotes the probability that both of messages to ci and 
c 2 from variable nodes v\ and v 2 in the cycle which connect 
to Ci and c 2 , respectively, and which are not ui, are erased. If 
Vi and v 2 are the same, i.e., if s 2 — 2s 2 — 1 = 1 holds, then 
r = eX'{P e (t - si - 1))/A'(1). Otherwise, 



X'(P e (t-si-l)) 



A'(l) 



where q' denotes the probability that both of messages to 
vi and v 2 from check nodes C3 and C4 in the cycle which 
connect to vi and v 2 , respectively, and which are not ci and 
c 2 , respectively, are erased. The probability q' is obtained in 
the same way as q. Summing ( fT9l , ( f20~b and ( |2TI ). we obtain 
Gi(i — Si, s 2 — 2si — 1). Hence, we obtain ( fToT ) and the proof 
that the contribution of neighborhood graphs of Type I and 
Type II with the parameters si and s 2 is F v (t, si, s 2 ) is done. 
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In almost the same way, the contributions of neighborhood 
graphs of Type III and Type IV are obtained as 

i/(l)A'(l)(A'(iy(l)) s - Sl - 2 £ (t + l) 

p"(l - Q £ (t - Sl )) 



x 9 (t, Sl ,l pn{1) 

xG 3 (t-si-l,s 2 -2s 1 -2) 
and that of Type V and Type VI are obtained as 

i(AW(l)) s e^pG!(M-l). 
Since the derivation is similar, the proof is omitted. 



IV. The limit of a(e, t) 

In this section, the limit values a(e, oo) :— lim^oo a(e,t) 
for regular ensembles are shown. The limit a(e, oo) has a sim- 
ple expression while the expression of a(e,t) is complicated 
and recursive. Empirically, the approximation using a(e, oo) 
instead of a(e, t) is accurate even for small blocklength if e is 
close to or 1, as will be observed in Section [VI] The proof 
of the following theorem is in Appendix lAl 

Theorem 3. For (7, r)-regular ensembles, let 



^e(oo) 


:= lim P e {t) 


Qe{oo) 


:= lim Q e (t) 

t— >oo 


V 


■= e(l - l)P e (< 


<1 


:= (r-l)(l- 


V 


:= e(l 


w 


:= (r - l)(r - 



\l-2 



y-2 



\r-3 



If pq < 1, the limit is 



a{t, oo) 



1 1 



21 -pq 



pq 



5 (oo)- 



1 



-q 2 v 



pq 



1 - pq 

(P £ (oo) - Qe(oo)) + 1 - P e (oo)Q e (oo) 

1 



2 QeM (l-M) 2 



1 



pq 



-(Qe(00) - P £ (oo)) + (1 - PM)(1 - Q e (00)) 



The quantity pq which appears in the condition of the 
theorem is the slope of the function of density evolution 
fde(x) — e\(l — p(l — x)), which described the evolution 
of Q t (t) in Lemma |2 at the largest fixed point x = Q e (oo) 6 



J-i 



and p(x) :- 



L . Hence, pq < 1 



[0, 1], where \(x) := x" 
is always satisfied, pq = 1 if and only if y = fd e (x) touches 
y = x at the largest fixed point. Such points of e includes the 
threshold cbp and the discontinuous point of the largest fixed 
point with respect to e. 

Especially, below the threshold, 

eAW(l) 



a(e, oo ) 



1 



2 1 - eA'(0)p'(l)' 



This quantity also appears in Q}, This fact implies that the 
following two limits are equal below the threshold for regular 
ensembles. 



lim lim n(Pb(n, e, t) — Pb(oo, e, t)) 

t— >oo n— >oo 



= lim lim n(Pt,(n, e, t) — Pb(oo, e, t)) 

n— foo t— loo 



The limit a(e, oo) for irregular ensembles is an open prob- 
lem. 



V. Uniform convergence under fixed number of 

ITERATIONS 

As mentioned in the introduction, the bit error probability 
after infinite number of iterations converges to a discontinuous 
curve with respect to erasure probability of a channel as the 
blocklength tends to infinity if A'(0) j o'(l)eBP < 1- Since 
the bit error probability for finite blocklength is continu- 
ous, the convergence is not uniform. Due to the lack of 
uniform convergence, an approximation (Q]i using asymptotic 
expansion with respect to blocklength is not accurate near 
the discontinuous points. Hence, for accurate approximation 
near discontinuous points, other approximations should be 
considered. The scaling-law-based approximation method was 
introduced by Amraoui et al. iflOl . Ifl3l for this purpose. 

In this section, we will show that the bit error probability af- 
ter a fixed number of iterations converges to a limit uniformly 
in contrast to the case of infinite number of iterations, which 
immediately implies that the approximation (f2j) is accurate for 
all e uniformly when the blocklength is sufficiently large. We 
have to show 



|P b (n 1 e,t)-P b (oo,e,t)| < C 



(22) 



where C = o(l)asn— > oo and C does not depend on e. The 
left-hand side of (l22l is bounded as 



|P b (n,e,t)-P b (oo,e,t)| = 



]T P„(G)P b (e,G) 



GST* 



< 



GST* 



P„(G)P b (e,G)+ MG)Pb(e,G) 
Ges t GeQt\(T t us t ) 

- P b (oo,e,t) 
Y P„(G)P b (e, G) - P b (oo, e, t) - /3(e, t)~ 

V P n (G)P b (e,G)- 7 (e,t)- 

+ |a(e,t)|i+ y Pn ( G ), (23) 
n Geff t \(T t us t ) 

From Lemma [T] the last term in the rightmost side of ( 1231 ), 
which depends on t but not on e, is ®(n~ 2 ). The first term in 
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the rightmost side of d23l ) is bounded as 

V P„(G)P b (e,G) -P b (oo,e,i) - (3(e,t)- 



GeTt 



V P„(G) - Poo(G) - - lim n (P„(G) - P oc (G)) 

^ * \ T) 71. — i-OO 

Ger t 



x P b (e,G) 



„(G) - Poo(G) - - lim n (P„(G) - P oc (G)) 



^ E 

GeT t 

Similarly, the second term in the rightmost side of d23T > is also 
bounded as 



J2 P„(G)P b (e,G)-7(e,t)- 

V (p„(G) - - \ lim nP„(G) 

* — ' \ n Ln— »oo 



GeS t 



n 

^ E 

Ge5 t 



Pb(e,G) 



P„(G) - - lim nP n (G) 

71 Ln— ^oo 



The above two bounds are 0(n 2 ) and are independent of e. 
Hence, (|23] > is upper bounded by 



|a(e,t)| 



1 



D 



where D = 0(n -2 ) depends on t but not on e. 

Since |a(e, t)| is continuous on e S [0, 1] and so bounded, 
we conclude that the bit error probability under a finite 
number of iterations converges to the limit uniformly as the 
blocklength tends to infinity. More accurately, we obtain 



P b (n, e, t) - P b (oo, e, t) - a{e, t) 



< D 



(24) 



from the above results. Equation (124b bounds an error of the 
approximation. However, this bound is available only under 
the assumption that the blocklength is sufficiently large so 
that all possible neighborhood graphs could be generated. In 
the next section, we observe via numerical calculations and 
simulations that the approximation is also accurate even if 
one cannot expect the assumption to be satisfied. 

VI. Numerical calculations and simulations 

In this section, we show calculation results of a(e,t) 
and a(e, oo) and show simulation results of n|Pb(ri, e, t) — 
P b (oo, e,t)\, the quantity which tends to |a(e,7;)| as n tends 
to infinity. 

The results of calculating a(e,t) for the (2, 3) -regular en- 
semble, the (3, 6)-regular ensemble and an irregular ensemble 
are shown in Fig. [2] Fig. [3] and Fig. |4j respectively. The 
coefficient a(e, t) seems to approach the limit a(e, oo) quickly 
where e is close to or 1. 

If e satisfies the two conditions A'(0)p'(l)e < 
(A'(l)p'(l)) -1 and e < e B p, then (3(e, t) tends to zero 
and 7(e, t) tends to the limit a(e, oo) as t tends to infinity. In 




Fig. 2. Calculation results for (2, 3)-regular ensemble. Thin curves show 
a(e,t) for t = 1,2,... ,8. Thick curve shows the limit a(e, oo). The 
threshold tap is 0.5. 




Fig. 3. Calculation results for (3, 6)-regular ensemble. Thin curves show 
a(e,t) for t = 1,2,..., 8. Thick curve shows the limit a(e, oo). The 
threshold e B p is about 0.42944. 




Fig. 4. Calculation results for an irregular ensemble. X(x) = 0.500x + 
0.153Z 2 + 0.112x 3 + 0.055a: 4 + 0.180a; 8 , p(x) = 0.492a; 2 + 0.508a; 3 . 
Thin curves show a(e, i) for t = 1, 2, . . . , 8. Thick curve shows the result 
with t = 50. The threshold eep is about 0.8. 
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Fig. 5. Simulation results for (2, 3)-regular ensemble. Blocklengths are 51, 
102 and 801. Number of iterations is 20. 
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Fig. 6. Simulation results for (3, 6)-regular ensemble. Blocklengths are 128, 
512 and 4096. Number of iterations is 5. 
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Fig. 7. Simulation results for an irregular ensemble. \(x) = 0.500a; + 
0.153a; 2 + 0.112a; 3 + 0.055a; 4 + 0.180x 8 , p(x) = 0.492a; 2 + 0.508a; 3 . 
Blocklengths are 360, 720 and 5760. Number of iterations is 20. 



this case, we can understand intuitively that dominant events 
of decoding error are events of errors of channel outputs in 
single-cycle neighborhood graphs consisting of variable nodes 
of degree 2 and check nodes, i.e., single-cycle stopping sets 
equivalent to single-cycle codewords also discussed in H. 
However, if A'(0)p'(l)e > (A'(l)p'(l))" 1 , which is the case 
when e > 0.25 in Fig. |2] and when e > 0.113 48 in Fig. H 
even below the threshold, f3(e,t) grows to — oo and j(e,t) 
grows to +oo exponentially in t. The reason of this large 
cancellation between /3(e,t) and 7(e,i) is not sufficiently 
understood. 

Because of the large cancellation, multiprecision arithmetic 
was necessary in our calculations to avoid cancellation errors 
in computation of a(e, t) with large t. 

Simulation results for the above ensembles are shown in 
Fig. [3] Fig. [6] and Fig. [7] respectively. For the (2, 3)-regular 
ensemble, the simulation results with n = 801 almost converge 
to a(e, t) for all e, as shown in Fig. [5] It is also the case with 
the irregular ensemble which has variable nodes of degree 
2 (Fig. [7), where the simulation results with n = 5760 
are observed to converge well to a(e,t) for all e. For the 
(3, 6)-regular ensemble (Fig. [5), the simulation results almost 
converge to a(e,t) for e > 0.25 at n = 4096. The agreement 
between simulation results and theoretical results is strange, 
since the pairs of the blocklength and the number of iterations 
are not suitable for density evolution technique in which 
one assumes that neighborhood graphs are tree with high 
probability. Indeed, the numbers of variable nodes in tree 
graphs are well above the total numbers of variable nodes in 
all the three cases considered: The number of variable nodes 
in a tree graph of depth 20 in the (2, 3)-regular ensemble is 
4194 302 which is much greater than the blocklength 801. 
The number of variable nodes in a tree graph of depth 5 in 
the (3, 6)-regular ensemble is 166 666 which is much greater 
than the blocklength 4096. The number of variable nodes in 
the minimum tree graph of depth 20 in the irregular ensemble 
is 4 194 302 which is much greater than the blocklength 5760. 
We have not succeeded in finding an appropriate explanation 
to the observed quick convergence. 

For (3, 6)-regular ensemble, the convergence to a(e, t) is not 
fast for e < 0.25. In the low-e region, dominant error events 
after infinite number of iterations are those induced by small 
stopping sets. (3, 6)-regular ensemble does not contain single- 
cycle stopping sets but contains three double-cycle stopping 
sets. When e is close to 0, unless the blocklength is sufficiently 
large, the bit error probability after a small number of iterations 
is almost the same as that after infinite number of iterations, 
since decoding will succeed after a few number of iterations 
with high probability. It is also the case when e is close to 1, in 
which case decoding will fail after a few number of iterations 
with high probability. Hence, in the low-e region, the bit error 
probability decays like 8(rt~ 2 ) rather than 0(n _1 ) unless the 
blocklength is sufficiently large. 

The well-established fact that the bit error probability at 
error floor is well approximated by (HJ [4| is interpreted as the 
statement that the bit error probability P^(n,e,t) when e is 
close to and A'(0) > is well approximated by Pb(oo, e, t) + 
a(e, oo)/n for large n. From the observed quick convergence 
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of a(e, t) to a(e, oo) and that of n(Pb(n, e, t) — Pb(oo, e, t)) 
to a(e,t) for e close to 1, the same statement is empirically 
valid when e is close to 1 as well. 



VII. Conclusion 

We have obtained the coefficient a(e,t) of the second 
dominant term in the asymptotic expansion of the bit error 
probability after a fixed number of iterations for irregular 
ensembles. Furthermore, we have obtained the limit a(e, oo) 
for regular ensembles. At last, we have confirmed that approx- 
imations using a(e, t) are accurate even for small blocklength. 

There are two important open problems. The first one is 
the large cancellation problem between f3(e,t) and 7(e,t). 
The underlying mechanism of this cancellation has not been 
understood sufficiently, so that, for example, we do not know 
whether similar cancellations occur in higher-order terms. 
The second one is the fast convergence problem of a(e,t). 
Simulation results show that the convergence to a(e, t) is very 
fast. This fact is strange since neighborhood graphs should 
include many cycles in moderate blocklengths. 

Some other works remain to be done. First, the limit 
a(e, oo) for irregular ensemble has not been derived. Sec- 
ond, optimization of finite-length irregular and expurgated 
ensembles given the number of iterations, blocklength, erasure 
probability, and allowable error probability, similar to the 
finite-blocklength optimization by Amraoui et al. |10|, [ 1 3 1 
for infinite number of iterations, is practically important. 
Third, derivation of the coefficients of higher-order terms 
n~ 2 ,n~ 3 , ... is an interesting problem. Fourth, other limits 
may also be important in practice. An example is the limit 
of blocklength and the number of iterations tending to infinity 
simultaneously. Assume t = clogn/ log(A'(l)p'(l)) for some 
constant c > 0. Then the probability of cycle-free neigh- 
borhood graphs tends to 1 for c < 1/2 and tends to for 
c > 1/2 lfl8l . It means that the cycle-free assumption is appli- 
cable only for c < 1/2, so that methods like density evolution 
under c > 1/2 are not available. At last, generalization of the 
methods to general channels and BP or other message passing 
decoders is important. There is a technically difficult problem 
due to reuse of messages from the same edges for calculation 
of the contributions of single-cycle neighborhood graphs. 



Appendix A 
Proof of Theorem[3] 

First, we show an alternative expression of a(e, t) for 
regular ensembles. The new expression is useful for proving 
Theorem[3] Furthermore, the expression does not require mul- 
tiprecision arithmetic which the previous expression requires 
in order to avoid the cancellation errors in the calculation of 
the sum f3(e, t) + 7(e, t). 

Lemma 4. For {l,r)-regular ensemble, a(e,t) is calculated 



t-l 2t 

a ( e ,t)=^2 T v(t,s 1 ,s 2 ) 

si = l s 2 =2si+l 

t-1 2t 



Sl =0 s 2 =2si+2 



s=l 



where 



T v (t, Sl ,s 2 ) := -Q £ (t + l)p'(l - Q e (t))eX"(P e (t - 8l )) 



H eX'(P e (t-k))p'(l-Q e (t-k))j 

x H 1 (t-s 1 ,s 2 - 2s l - 1) 



. fe=i 



T c {t, Sl, 8 2 ) ~ ^Qe{t + l)p"{l - Qe{t - Si)) 

x n eA '( p ^ - k - - - fe )) 



, fe=0 



x H 3 (t -si - 1,8 2 - 2sx - 2) 



T r (t,s) :=^eX'{P e {t))H x {t,s-l) 



Ht{t,s) := 

P '(i)(i~p e m, if 8=o 

P '{l-Q e {t)) 2 H 2 {t- l,s-l), ifs>t 

2(p'(l)-p'(l-Q e (t)))(l-Pe(t-s)) 
x IliZl P'{1 - QS - k))e\'{PS -k-1)) 
+p'(l - Q £ (t)) 2 H 2 (t - 1, s - 1), otherwise 

{e\>(P e (t))~\i(l)Q t (t + l) 2 , ifs = 
2[ ">- lieX'iMtWH^s-l), 



H 3 (t,s) := 

e\'(P c (t)) - X'(l)Q e (t + 1)(2 - Q £ (t + 1)), if s = 
-(eA'^))) 2 ^,*-!), ifs>t 
2eX'(P e (t))(l-P e {t-s)) 
x n^o - Qe(t - k))eX'(P e (t -k-1)) 
-(eA'(F e (t))) 2 iJi(t, s - 1), otherwise. 

and where X(x) ~ x 1 ^ 1 and p(x) — x r ~ 1 . 

Outline of proof of Lemma® For (I, r)-regular ensemble, 
the cycle-free neighborhood graph is unique. The coefficient of 
rT 1 in the probability of the unique cycle-free neighborhood 
graph is 

2 V ' 1 - (I - l)(r - 1) U A ;/ 
Hence, f3(e,t) for (I, r)-regular ensemble is obtained as 

P(e,t) = --l(r-l) 1 _ {l _ 1){r _ 1) 

x eP e {t) 1 . 
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It is decomposed as follows. 



1 



P(e,t) = -eP e (t) 1 
' t-i it 

E E A"(i)p'(i)(A'(i) P '(i)r- si - 2 

.si = l 2si + l 
t-l 2t 

+ E E P"(1)A'(1)(A'(1) P '(1)) S2 -^- 2 

Sl =0s 2 =2si+2 

2t 

+ E(A'(l)p'(l)) J 

s=l 

Hence, a(e, t) is calculated as 

t-l 2t 

«(m) = £ E 

si=l 2si+l 

F v (t, Sl ,s 2 ) l -\"{l)p\l){\'{l)p'(l)y^~ 2 eP t {t) 1 

t-l 2t 

+ £ E 

s 1= 0s 2 =2si+2 
2t , 

+ J2(Fr(t,s) - -(X'(l)p'(l)yeP e (t) 1 

s=l \ 

t-l 2t t-l 2t 

= E E T u(Ml,S 2 ) + £ E r c(i,Sl,S2) 
si = ls 2 =2si + l si=0s 2 =2si+2 
2t 

+£r r (*,*)■ 

s=l 

We omit calculations of T v (t, si, s 2 ), r c (i, si, s 2 ) and T r (t, s). 

■ 

Proof of Theorem \3] After some calculations, we obtain 

t-l 2t 



lim lim T„(u, Si,s 2 ) 

t— >-oo * — ' * — ' it— >-oo 



SI— 1 S2— 2si+l 



1 



1 -pq 



(P 6 (00) " Qe(00)) + 1 - P £ (oo)Q 6 (oo) 



t-l 2t 



lim E E lim r c (u, si,s 2 ) 



f— J-oo f ■* ' * u-^oo 
Sl =0 s 2 =2si+2 



(Q e (00) - P e (00)) + (1 - P e (oo))(l - Q e (oo)) 

l-pq 
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lim > lim T r (u,s) = — pq 



1 — pq 



(P e (oc) - Q e (oo)) + 1 - P e (oo)Q £ (oo) 



If there exist T v (si,S2), T c (si,S2) and T r (s) such that 
\T v (t, Sl ,s 2 )\<f v ( Sl ,s 2 ), for alii 

|Tc(Mi,S2)|<T c (si,*3), for all t 

\T r (t,s)\<T r (s), for all t 

and such that 



t-i 



2t 



lim , 

t— >oo — ' 

si=l s 2 =2si + l 

t-l 2t 

lim V V 

t->00 ^ ^ 

s 1= s 2 =2si+2 



E T »( S 1' S 2) < OO 

r c (si, s 2 ) < 00 



2t 

lim \ T r (s) < 00 

t— >oo — ' 

s=l 

then Theorem [3] is a consequence of Lebesgue's dominated 
convergence theorem. If eA'(P e (oo))//(l — Q e (oo)) < 1, there 
exists 6 > such that 

e(A'(P e (oo)) + 5)0/(1 - Q e (oo)) + S) < 1. 

On the other hand, 

\X'(P e (t))-X'(P e (oo))\<6 (25) 
|p'(l-Q e (i))-p'(l-Q e (oo))[ <6 (26) 

for all but finite t. One can therefore take Pu(si,s 2 ), 
T c (si, s 2 ) and T r (s) satisfying the above conditions by replac- 
ing A'(P e (t)) and p'(l-Q e (t)) in T v (t,s u s 2 ), T e (t,a 1 ,8 2 ) 
and T r (i,s) with A'(P e (oo)) +5 and p'(l - Q e (oo)) + <5, re- 
spectively, and multiplying them with an appropriate constant 
in order to take into account the fact that the bounded number 
of A'(PeO)) and p'(l - QS)) in T v (t, Sl ,s 2 ), T c (t, Sl ,s 2 ) 
and T r (t, s) do not satisfy (l25T l and 
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